REMARKS 

This application has been reviewed in light of the Office Action mailed on April 9, 
2004. Claims 1-38 are pending in the application with Claims 1,11,21 and 30 being in 
independent form. 

(1) In the Office Action, Claims 1-7, 9, 11-17, 19, 21-26, 28, 30-35 and 37 were 
rejected under 35 U.S.C. § 102(b) as being anticipated by the article entitled "Color 
SuperHistograms for Video Representation", written by Dimitrova et al. 

(2) In the Office Action, Claims 8, 10, 18, 20, 27, 29, 36 and 38 were rejected 
under 35 U.S.C. § 103(a) over the article entitled "Color SuperHistograms for Video 
Representation", written by Dimitrova et al. 

Applicant appreciates the courtesy granted to Applicant's attorney, Michael A. 
Scaturro (Reg. No. 51,356), during a telephonic interview conducted on July 1, 2004. 
During the telephonic interview, the cited reference in the instant Office Action, namely, 
"Color SuperHistograms for Video Representation", written by Dimitrova et al. was 
discussed. In particular, Applicant's attorney presented reasons stating why the claims 
are not anticipated by and are patentably distinguishable over the article. 

In the interview, Applicant's attorney explained why the cited article does not 
teach or disclose the steps of creating a visual summary, as alleged by the Examiner in the 
instant Office Action. It was argued that the article only teaches how to create a 
superhistogram which is a preliminary step towards the creation of a visual summary. 
Superhistograms do not by themselves represent a visual summary. 
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During the telephonic interview, the Supervisory Examiner stated that the creation 
of a superhistogram could be construed to represent, in a broad interpellation, a kind of 
visual summary. The Examiner further suggested that an amendment to the claims may 
be needed to better define Applicant's invention and to patentably distinguish Applicant's 
invention over the cited article. Applicant's have so amended the claims in accordance 
with the Examiner's recommendation. 

During the interview, Applicant's attorney agreed to attempt to repeat the 
explanation distinguishing a superhistogram from a visual summary as presented during 
the interview in the instant response. Accordingly, the following recitation is an attempt 
on the part of the Applicant's Attorney to repeat the substance of the explanation 
provided. 

During the telephonic interview, Applicant's Attorney's explanation began with a 
broad overview of the method steps required to create a visual summary in accordance 
with the principles of the invention, as recited in the specification of the patent 
application. The method steps, as discussed, are as follows: 

1. receivings video signals. 

2. formatting the video signals into frames. 

3. identifying keyframes. 

4. extracting frame signatures from the keyframes. 

5. using the extracted frame signatures to filter the identified keyframes. 

6. generating color information from the filtered keyframes. 

7. derive superhistograms from the color information 

In order to create a visual summary from a superhistogram: 

8. selecting representative keyframes from the superhistograms 

9. creating a compact visual summary using the selected representative keyframes. 
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Steps 8 and 9 collectively represent the two steps required to create a visual 
summary from a superhistogram , in accordance with the principles of the invention. 
Applicant's Attorney emphasized to the Examiner that, steps 8 and 9 are not taught by the 
cited article . The article is directed to step 7, i.e., deriving superhistograms from the 
color information. The article makes no reference whatsoever to creating a visual 
summary, as recited in Independent claims 1, 11,21 and 30. 

It is instructive to briefly review the article. The article presents a novel method for 
computing superhistograms for applications including video classification and retrieval 
methods such as video editing, studio archival, digital libraries, consumer products and 
web crawling (see abstract). In accordance with the method disclosed in the article for 
computing superhistograms, video content is systematically analyzed to ascertain 
thematic visual elements (e.g., color) to assist in characterizing the video content using 
cumulatively averaged histograms (i.e., superhistograms). The article states (in the 
abstract) that certain genres of television programs have a similar look and feel as far as 
their color composition is concerned. The article attempts to exploit this similar look and 
feel by utilizing superhistograms to reveal the color signatures (i.e., the look and feel) of 
the input video sequence being analyzed. 

The article illustrates the result from experiments performed, whereby a number of 
superhistograms were created for 4 television program genres. It was determined that 
there is a drastic difference between sitcoms and news. However, the two 
superhistograms of the sitcoms look somewhat similar. As shown in Fig. 3, each 
superhistogram is made up of the 3 family histograms. In Fig. 3, part d, the 
superhistogram for CNN News contains a significant amount of blues in addition to the 
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other color distributions collectively representing a color signature for CNN news in a 
compact style. Similarly, Fig. 3, parts a and b, show superhistograms for the programs 
"Just Shoot Me" and "Seinfield", respectively, which both include a significant amount of 
browns representative of their color signatures. It is instructive to point out that 
superhistograms are embodied in this context as color signatures (bins) of the respective 
video sequences. 

For purposes of distinguishing a visual summary of the invention from a 
superhistogram of the article, it is important to provide a precise definition of a 
superhistogram. 

Referring to section 2.4 of the article, it states that given a video segment, we 
choose the top n largest families in order to represent the video within family histograms. 
As such, a superhistogram is comprised of the top n largest family histograms. Given that 
a superhistogram is a kind of superset of family histograms, it becomes incumbent to 
define a family histogram. This definition is provided at Page 314, second paragraph, 
wherein it is stated that color histograms are computed for individual shots and then 
merges the histograms into a single cumulative histogram called a family histogram. 

While this definition is accurate, it is not all inclusive. Referring now to section 2.3 
of the article, it is stated that The family histogram is a data structure consisting of 
pointers to each of the constituent histograms and frame numbers; a merged family 
histogram. A family histogram may therefore be defined as both a cumulative color 
histogram and a data structure. 

Given that a superhistogram is a superset of family histograms, the above 
definition is applicable to a superhistogram. That is, in one sense, it is a single cumulative 
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color histogram of family histograms comprised of the top n largest family histograms, 
and, in another sense, it is also a data structure which points to the data structures of the n 
largest family data structures. 

In light of the above definitions, the article is distinguishable from the invention in 
the following way. In the article, the first aspect of the superhistogram is utilized. That 
is, Fig. 3 represents the color union of the top 3 family histograms. However, the 
invention utilizes the second aspect of a superhistogram. That is, a visual summary is 
derived from selected keyframes of the superhistogram. Selecting keyframes incorporates 
the data structure aspect, whereby the superhistogram is a data structure pointing to all of 
the constituent frames from which "selected keyframes" may be chosen. Further, 
keyframe selection from the superhistograms may depend on different criteria, as taught 
on Page 16 of the specification. 
Page 16 states: 

Controller 130 executes keyframe selection application 250 to 
select representative keyframe images for each superhistogram. The 
selected representative keyframe images can be selected from either (1) the 
first image in the family histogram, or (2) the most meaningful image in 
the superhistogram, or (3) a randomly chosen image or an image that is 
closest to the cluster (family) center. 

Pages 16 and 17 of the specification teach the second step, step 9, for creating a 
visual summary from the superhistograms: 

Visual summary application 260 then creates a compact visual summary 
using the selected keyframe images. 
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It is therefore respectfully submitted that a superhistogram is both a color union of 
frames of a video sequence and a data structure from which certain representative 
keyframes may be selected in accordance with different selection criteria. 

The article utilizes the first aspect of a superhistogram, i.e., the color union of 
frames, for applications such as video classification and retrieval methods such as video 
editing, studio archival, digital libraries, and consumer products. While the invention 
utilizes the data structure aspect of the superhistograms to perform frame selection of 
representative keyframes to be included in a visual summary. For example, the method 
may decide to select video frames {1, 23 and 84} to be included in the visual summary 
where video frame 1 is selected from family histogram 1, and video frame 23 is selected 
from family histogram 2 and video frame 84 is selected from family histogram 3. Where 
family histograms 1-3 collectively constitute the superhistogram and where family 
histogram 1 may comprise video frames {1, 12, 44 and 92} and family histogram 2 may 
comprise video frames {3, 23, 33, 49 and 87} and family histogram 3 may comprise 
video frames {11, 28, 39, 55, 74 and 84}. 

Independent Claims 1,11,21 and 30 have been amended herein to better define 
Applicant's invention and to patentably distinguish Applicant's invention over the cited 
article. Claims 1,11,21 and 30 now recites limitations and/or features which are not 
disclosed or suggested by the article. In particular, Claims 1, 1 1, 21 and 30 recite in part, 
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...using said frame signatures and said superhistograms to select certain 
keyframe images to create a compact visual summary of said video material. [Emphasis 
Added] 



Applicant respectfully submits that Claims 1,11,21 and 30 as amended 
hereinabove, is not anticipated by the cited article for at least the reasons provided above. 

For the reasons given above, the cited reference does not anticipate the subject 
matter of Claims 1,11,21 and 30. Accordingly, applicant respectfully requests that the 
rejection under 35 U.S.C. §102(b) with respect to Claims 1,11,21 and 30 and allowance 
thereof is respectfully requested. 

Additionally, Claims 2-7, 12-17 and 19, 22-26 and 28 and 31-35 and 37 depend 
from independent Claims 1, 11,21 and 30, respectively and therefore contain the 
limitations of Claims 1, 11,21 and 30. Hence, for at least the same reasons given for 
Claims 1, 11, 21 and 30, Claims 2-7, 12-17 and 19, 22-26 and 28 and 31-35 and 37 are 
believed to be allowable over the cited reference. Accordingly, withdrawal of the 
rejection under 35 U.S.C. §102(b) with respect to Claims 2-7, 12-17 and 19, 22-26 and 28 
and 31-35 and 37 and allowance thereof is respectfully requested. 

Claims 8, 10, 18, 20, 27, 29, 36 and 38 were rejected under 35 U.S.C. §103(a) over 
the article entitled "Color SuperHistograms for Video Representation", written by 
Dimitrova et al. 

Claims {8, 10}, {18, 20), {27, 29), and {36, 38} depend from independent Claims 
1,11,21 and 30, respectively, and therefore contain the limitations of Claims 1,11,21 
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and 30. Hence, for at least the same reasons given for Claims 1, 11,21 and 30, Claims 
{8, 10}, {18, 20), {27, 29), and {36, 38}are believed to be allowable over the cited 
reference. Accordingly, withdrawal of the rejection under 35 U.S.C. § 103(a) with respect 
to Claims 8, 10, 18, 20, 27, 29, 36 and 38 and allowance thereof are respectfully 
requested. 

In view of the foregoing amendments and remarks, it is respectfully submitted that 
all claims presently pending in the application, namely, Claims 1 - 38 are believed to be 
in condition for allowance and patentably distinguishable over the art of record. 

If the Examiner should have any questions concerning this communication or feels 
that an interview would be helpful, the Examiner is requested to call Dicron Halajian, 
Esq., Intellectual Property Counsel, Philips Electronics North America Corp., at 914-333- 



9607. 



Respectfully submitted, 




Dicran Halajian, Reg. 39fn)3 
Attorney 
(914) 333-9607 
July 8, 2004 
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